Applying One Reason Decision Making: The Prioritization of Literature Searches

When researchers start doing a literature search, they often initially find an unmanageably large number of potentially relevant articles, and are forced to refine their searches. Faced with this information overload, prioritization offers an alternative to search refinement, by ordering the articles so that the ones most likely to be relevant are at the top of the list. Effective prioritization relies on having a good model of human decision making that can learn from the way people decide whether articles are relevant, and make predictive decisions about which of the remaining articles will be relevant. In this paper, we develop and evaluate two psychological decision making models for prioritization. One is a ‘rational’ model that weights and combines all of the available information to make decisions. The other is a ‘one reason’ model that uses limited time and information. The models are evaluated in an experiment where users evaluated every article returned by PsycINFO for a number of different research topics. The results show that both models achieve a level of prioritization that significantly improves upon the default ordering of PsycINFO. The one reason model is also shown to be superior to the rational model, especially when there are only a few relevant articles. The implications of the results for developing prioritization systems in applied settings are discussed, together with implications for the general modeling of human decision making. In particular, it is argued that there is evidence users do employ one reason decision making when selecting relevant articles, but that the development of better representational techniques is required before prioritization can be used in real-world applications. ∗This research was supported by the Australian Defence Science and Technology Organisation, and by the University of Adelaide Faculty of Health Sciences B3 Scheme. Correspondence should be addressed to: Michael D. Lee, Department of Psychology, University of Adelaide, SA 5005, AUSTRALIA. Telephone: +61 8 8303 6096, Facsimile: +61 8 8303 3770, E-mail: [email protected], URL: http://www.psychology.adelaide.edu.au/members/staff/michaellee/homepage